Papers with contrastive learning approach
Can Sequence-to-Sequence Transformers Naturally Understand Sequential Instructions? (2023.starsem-1)
Copied to clipboard
| Challenge: | Using a limited annotation budget, we can greatly improve the performance on intermediate steps with a drop in final-step performance. |
| Approach: | They propose to use a pre-supervised sequence-to-sequence transformer to provide training signals on intermediate steps with zero gold supervision instead of only final-step supervision to improve performance. |
| Outcome: | The proposed model significantly improves on intermediate steps with a drop in final-step performance on one subtask, but also shows decreased performance on another subtask. |
Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing NLP methods treat morality as binary, ranging from right to wrong. |
| Approach: | They propose to build a pluralist moral sentence embedding space using contrastive learning methods to examine relationships among moral elements. |
| Outcome: | The proposed method shows that pluralism can be captured in an embedding space. |
CoLeM: A framework for semantic interpretation of Russian-language tables based on contrastive learning (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for semantic interpretation of tables in Russian lack explicit semantics . tabular data are one of the key formats for presenting structured information in various domains ranging from scientific research to business analytics. |
| Approach: | They propose a contrastive learning approach to minimize reliance on manual labeling and enhance the accuracy of column annotation for rare semantic types. |
| Outcome: | The proposed method minimizes reliance on manual labeling and improves column annotation accuracy for rare semantic types. |
Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing models memorize procedures from context and rely on shallow heuristics to solve MWPs. |
| Approach: | They propose a contrastive learning approach where the neural network perceives the divergence of patterns. |
| Outcome: | The proposed method greatly improves performance in monolingual and multilingual settings. |
Grammatical Error Correction with Contrastive Learning in Low Error Density Domains (2021.findings-emnlp)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) is a text generation task . performance on low error density domains where texts written by native speakers can be improved. |
| Approach: | They propose a contrastive learning approach to encourage the GEC model to assign a higher probability to a correct sentence while reducing the probability of incorrect sentences that the model tends to generate. |
| Outcome: | The proposed approach significantly improves the performance of GEC models in low error density domains. |
Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data. |
| Approach: | They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning. |
| Outcome: | The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset. |
Lexical Knowledge Internalization for Neural Dialog Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialog models ignore the knowledge that resides in people's minds during a conversation. |
| Approach: | They propose to integrate lexical knowledge internally into the model's parameters instead of further conditioning them on external knowledge . they adopt contrastive learning approach and use a dictionary-based token-level lexicon retriever that requires only weak supervision. |
| Outcome: | The proposed model can relate J.K Rowling to Khalsa Aid with the knowledge retrieved from Wikipedia. |
SharedCon: Implicit Hate Speech Detection using Shared Semantics (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies suggest that classifying hateful posts in a binary manner may not address nuanced task of detecting implicit hate speech. |
| Approach: | They propose a contrastive learning approach that leverages shared semantics among data to detect implicit hate speech. |
| Outcome: | The proposed approach is based on a clustering-based contrastive learning approach with human-written implications or machine-generated augmented data. |
Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach (P19-1)
Copied to clipboard
| Challenge: | Existing methods for reducing word omission errors in neural machine translation are prone to omit essential words on the source side. |
| Approach: | They propose a contrastive learning approach to reduce word omission errors in NMT by omitting words. |
| Outcome: | The proposed approach achieves better translation performance than baseline methods on Chinese-to-English, German-to English, and Russian-toEnglish translation tasks. |
Contrastive Learning for Inference in Dialogue (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent large language models show remarkable advances in inference tasks, but their performance in inductive reasoning is far behind deductive reasoning. |
| Approach: | They propose to use negative samples to analyze inferences based on the semantic information gap between dialogue contexts and desired inference. |
| Outcome: | The proposed model improves inference generation by feeding negative samples to the models. |
Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Prediction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Modern review helpfulness prediction systems focus on polishing cross-modal representations and suffer from inferior optimization. |
| Approach: | They propose a method to polish cross-modal relation representations by learning mutual information through contrastive learning. |
| Outcome: | The proposed framework outperforms baselines and achieves state-of-the-art results on two publicly available datasets. |
Mitigating Spurious Correlations via Counterfactual Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
Fengxiang Cheng, Chuan Zhou, Xiang Li, Alina Leidinger, Haoxuan Li, Mingming Gong, Fenrong Liu, Robert Van Rooij
| Challenge: | Existing methods to distinguish causally related words from spurious correlations are limited by the number of causally correlated words in a sentence. |
| Approach: | They propose to use probabilistic probability of necessity and probability of sufficiency to identify causal relationships rather than spurious correlations between words and class labels. |
| Outcome: | The proposed method is based on a contrastive learning approach name CPNS and is validated on public datasets. |